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Abstract 

In  the  analog  VLSI  implementation  of  neural  systems,  it  is  sometimes  convenient  to  build 
lateral  inhibition  networks  by  using  a  locally  connected  on-chip  resistive  grid.  A  serious 
problem  of  unwanted  spontaneous  oscillation  often  arises  with  these  circuits  and 
renders  them  unusable  in  practice.  This  paper  reports  a  design  approach  that 
guarantees  such  a  system  will  be  stable,  even  though  the  values  of  designed  elements  in 
the  resistive  grid  may  be  imprecise  and  the  location  and  values  of  parasitic  elements  may 
be  unknown.  The  method  is  based  on  a  rigorous,  somewhat  novel  mathematical 
analysis  using  Tellegen's  theorem  and  the  idea  of  Popov  multipliers  from  control  theory. 

It  is  thoroughly  practical  because  the  criteria  are  local  in  the  sense  that  no  overall 


analysis  of  the  interconnected  system  is  required,  empirical  in  the  sense  that  they 
involve  only  measurable  frequency  response  data  on  the  individual  cells,  and  robust  in 
the  sense  that  unmodelled  parasitic  resistances  and  capacitances  in  the  interconnect 
network  cannot  affect  the  analysis.  / 


^grajBWioN  8TATEMTW 

Approved  for  public  relaa a 
_  Djatributloa  Unlimited 


Microsystems  Massachusetts  Cambridge 

Research  Center  Institute  Massachusetts 

Room  39-321  of  Technology  02139 


*8  K 


Telephone 

(617)253-8138 


12  0  40 


Acknowledgements 

To  appear  in  Proc.  IEEE  Int.  Svmp.  on  Circuits  and  Systems.  Helsinki,  Finland,  June 
1988.  This  work  was  supported  in  part  by  Defense  Advanced  Research  Projects  Agency 
through  the  Office  of  Naval  Research  under  ARPA  Order  No.  3872,  Contract  numbers 
N00014-80-C-0622  and  N00014-87-K-0825. 


Author  Information 

Wyatt:  Department  of  Electrical  Engineering  and  Computer  Science,  MIT,  Room  36-864, 
Cambridge,  MA  02139,  (617)  253-6718;  Standley:  Department  of  Electrical  Engineering 
and  Computer  Science,  MIT,  Room  36-848,  Cambridge,  MA  02139,  (617)  25343454. 


Copyright®  1988  MIT.  Memos  in  this  series  are  for  use  inside  MIT  and  are  not 
considered  to  be  published  merely  by  virtue  of  appearing  in  this  series.  This  copy  is  for 
private  circulation  only  and  may  not  be  further  copied  or  dtetributed,  except  for 
government  purposes,  if  the  paper  acknowledges  U.  S.  Government  sponsorship. 
References  to  this  work  should  be  either  to  the  puNshed  version,  if  any,  or  in  the  form 
“private  communication.’'  For  information  about  the  ideas  expressed  herein,  contact  the 
author  directly.  For  information  about  this  series,  contact  Microsystems  Research 
Center,  Room  39-321,  MIT,  Cambridge,  MA  02139;  (817)  253-8138. 


CIRCUIT  DESIGN  CRITERIA  FOR  STABLE  LATERAL  INHIBITION  NEURAL  NETWORKS 


J.L.  WYATT,  Jr.  And  D.L.  ST  ANDIE  V 


Department  of  Electrical  Engineering  and  Computer  Science 
Massachusetts  Institute  of  Technology 
Cambridge ,  Massachuaetts  02139 


ABSTRACT 

In  the  analog  VLSI  implementation  of  neural 
systems,  it  is  sometimes  convenient  to  build 
lateral  inhibition  networks  by  using  a  locally 
connected  on-chip  resistive  grid.  A  serious  prob¬ 
lem  of  unwanted  spontaneous  oscillation  often  arises 
with  these  circuits  and  renders  them  unusable  in 
practice.  This  paper  reports  a  design  approach  that 
guarantees  such  a  system  will  be  stable,  even  though 
the  values  of  designed  elements  in  the  resistive 
grid  may  be  imprecise  and  the  location  and  values  of 
parasitic  elements  may  be  unknown.  The  method  is 
based  on  a  rigorous,  somewhat  novel  mathematical 
analysis  using  Tellegen's  theorem  and  the  idea  of 
Popov  sailtipliers  from  control  theory.  It  is 
thoroughly  practical  because  the  criteria  are  local 
in  the  sense  that  no  overall  analysis  of  the  inter¬ 
connected  system  is  required,  empirical  in  the 
sense  that  they  involve  only  measurable  frequency 
response  data  on  the  individual  cells,  and  robust 
in  the  sense  that  unmodelled  parasitic  resistances 
and  capacitances  in  the  interconnect  network  cannot 
affect  the  analysis. 

I .  INTRODUCTION 

The  term  "lateral  inhibition"  first  arose  in 
neurophysiology  to  describe  a  common  form  of  neural 
circuitry  in  which  the  output  of  each  neuror.  in 
some  population  is  used  to  inhibit  the  response  of 
each  of  its  neighbors.  Perhaps  the  best  understood 
example  is  the  horisontal  cell  layer  in  the  verte¬ 
brate  retina,  in  which  lateral  inhibition  simul¬ 
taneously  enhances  intensity  edges  and  acts  as  an 
automatic  gain  control  to  extend  the  dynamic  range 
of  the  retina  as  a  whole  (1).  The  principle  has 
been  used  in  the  design  of  artificial  neural  system 
algorithms  by  Kohonen  12]  and  others  and  in  the 
electronic  design  of  neural  chips  by  Carver  Mead  et. 
al.  13,41. 

In  the  VLSI  implementation  of  neural  systems, 
it  is  convenient  to  build  lateral  inhibition  net¬ 
works  by  using  a  locally  connected  on-chip  resistive 
grid.  Linear  resistors  fabricated  in,  e.g.,  poly¬ 
silicon,  yield  a  very  compact  realisation,  and  non¬ 
linear  resistive  grids,  eade  from  MOS  transistors, 
have  been  found  useful  for  image  segmentation.  (4, 

51 .  Networks  of  this  type  can  be  divided  into  two 
clessesi  feedback  sy stasis  end  feedforward-only 
eyetesM.  In  the  feedforward  case  one  eet  of 
amplifiers  imposas  signal  voltages  or  currents  on 
the  grid  end  another  set  reeds  out  the  resulting 


response  for  subsequent  processing,  while  the  sane 
amplifiers  both  "write*  to  the  grid  end  "read"  from 
it  in  a  feedback  arrangement.  Feedforward  networks 
of  this  type  ere  inherently  stable,  but  feedback 
networks  need  not  be. 

A  practical  example  is  one  of  Carver  Mead's 
retina  chips  (31  that  achieves  edga  enhancement  by 
means  of  lateral  inhibition  through  a  rasistive 
grid.  Figure  1  shows  a  tingle  cell  in  a  continuous- 
time  version  of  this  chip.  Note  that  the  capacitor 
voltage  is  affected  both  by  the  local  light  intensity 
incident  on  that  cell  and  by  the  capacitor  voltages 
on  neighboring  cells  of  identical  design.  Any  cell 
drives  its  neighbors,  which  drive  both  their  dis¬ 
tant  neighbors  end  the  original  cell  in  turn.  Thus 
the  necessary  ingredients  for  instability-active 
elements  and  signal  feedback — are  both  present  in 
this  system,  end  in  fact  the  continuous-time  version 
oscillates  so  badly  that  the  original  design  is 
scarcely  ueable  in  practice  with  the  lateral  inhi¬ 
bition  paths  enabled.  [6]  Such  oscillations  can 


Figure  1.  This  photoreceptor  and  signal  processor 
circuit,  using  two  N3S  transconductance  amplifiers, 
realises  leterel  inhibition  by  cosmunicating  with 
similar  units  through  a  resistive  grid. 

readily  occur  in  any  resistive  grid  circuit  with 
active  element*  and  feedback,  even  when  each 
individual  cell  is  quite  stable.  Analysis  of  the 
conditions  of  instability  by  straightforward  methods 
appears  hopeless,  since  the  number  of  simultaneously 
active  feeAack  loops  is  enormous. 

This  paper  reports  s  practical  design  approach 
that  rigorously  guarantees  such  a  system  will  be 
stable.  The  very  simplest  version  of  the  idea  is 
intuitively  obvious i  design  each  individual  cell  so 
that,  although  internally  active,  it  acts  like  a 
passive  system  as  seen  from  the  resistive  grid,  in 


to  parasitic  elements  not  captured  by  tha  simple 
nodal*  in  rig.  2.  (ter  preliminary  analysis  of  a 
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circuit  theory  language .  the  design  goal  her*  is 
that  each  cell's  output  iapedanc*  should  be  a 
positive-real  [7]  function.  This  is  sonatinas  not 
too  difficult  in  practice)  we  will  show  that  the 
original  network  in  rig.  1  satisfies  this  condition 
in  the  absence  of  certain  parasitic  elements.  Mora 
important,  perhaps!  it  is  a  condition  one  can  verify 
experisan tally  by  frequency-response  msesurements . 

It  is  physically  apparent  that  a  collection 
of  cells  that  appear  passive  at  their  terminals  will 
for*  a  stable  system  when  interconnected  through 
a  passive  medium  such  as  a  resistive  grid.  The 
research  contributions!  reported  here  in  si  usury 
form!  are  i)  a  dsawnstration  that  this  passivity  or 
positive-real  condition  is  much  stronger  than  we 
actually  need  and  that  weaker  condition*!  more  easily 
achieved  in  practice!  suffice  to  guarantee  stability 
of  the  linear  network  model ,  and  ii)  an  extension  to 
the  nonlinear  domain  that  furthermore  rules  out 
large-scale  oscillations  under  certain  conditions. 

11.  FIRST-ORDER  LINEAR  ANALYSIS  OF  A 
SINGLE  CELL 

We  begin  with  a  linear  analysis  of  an  elemen¬ 
tary  model  for  the  circuit  in  Fig.  1.  For  an  initial 
approximation  to  the  output  admittance  of  the  cell 
we  simplify  the  topology  (without  loss  of  relevant 
information)  and  us*  a  naive'  nodal  for  the  trans- 
conductance  amplifiers,  as  shown  in  Fig.  1. 


Figure  2.  Simplified  network  topology  and  trans- 
conductance  amplifier  model  for  the  circuit  in  Fig. 
1.  The  capacitor  in  Fig.  1  has  been  absorbed  into 
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Straightforward  calculations  shew  that  the 
output  admittance  is 


Y  (s) 


l9B,2**021+a  C°2] 


9*^9ai2^oi 
(l+s  Roj^o^ 


(1) 


This  is  a  positive-real,  i.e.,  passive,  admittance 
that  could  always  be  realized  by  a  network  of  the 
form  shown  in  Fig.  3,  where 


<g*2+ROj *  »  *2* ^mi^B>2R°l^  *  4rK*  L  “  coi/,9mi9«2' 

Although  the  original  circuit  contains  no 
inductors,  the  realisation  has  both  capacitors  and 
inductors  snd  thus  is  cspsbl*  of  daspsd  oscillations. 
Nonetheless,  if  th*  transamp  model  in  Fig.  2  were 
perfectly  accurate,  no  network  created  by  inter¬ 
connecting  such  cells  through  a  rasistivs  grid  (with 
parasitic  capacitances)  could  inhibit  sustained 
oscillations  since  all  the  elements  ara  passive. 

For  eleawnt  values  that  may  be  typical  in  practice, 
th*  modal  in  Fig.  3  has  a  lipitly  damped  resonance 
around  1  KXx  with  a  Q  •  10.  This  disturbingly  high 
Q  suggests  that  the  cell  will  be  highly  sensitive 


Figure  3.  Passive  network  realisation  of  tha  out¬ 
put  admittance  (aq.  (1))  of  tha  circuit  in  Fig.  2). 

much  more  complex  model  extracted  from  a  physical 
circuit  layout  craatad  in  Carver  Mead's  laboratory 
Indicated  that  tha  output  impedance  will  not  be 
passive  for  all  values  of  tha  transamp  bias  currents 
But  a  definite  explanation  of  th*  instability  awaits 
a  more  careful  circuit  modelling  effort  and  perhaps 
th*  design  of  an  on-chip  impedance  measuring 
inetruswnt. 

III.  STABILITY  OF  A  LINEAR  MODEL  FOR  THE  NETWORK 

Transistor  parasitics  and  layout  parasitics 
will  cause  tha  output  admittance  of  the  individual 
active  cells  to  deviata  from  tha  form  given  in  eq. 
(1)  and  Fig.  3,  and  any  very  accurate  model  will 
necessarily  be  quit*  high  order.  The  following 
theorem  shows  what  sort  of  deviations  we  can  allow 
and  still  guarantee  that  the  network  is  stable. 

Terminology 

The  term  closed  right  half  plane  refers  to  the 
eet  of  complex  numbers  s  ■  o  ♦  ju  with  o>0  and  the 
term  closed  third  quadrant  refers  to  the  set  of 
complex  numbers  with  o<0  and  u<0.  A  natural 
frequency  is  a  complex  frequency  sD  such  that, 
whan  all  branch  impsdancas  and  admittances  are 
evaluated  at  s0.  there  exists  a  nonzero  solution 
for  th*  complex  branch  voltage*  {Vk}  and  currents 
{Ik>. 

Theorem  1 


-  Consider  a  linear  network  of  arbitrary  topology 
consisting  of  any  number  of  positive  2-tezminal 
resistors  and  capacitors  end  of  N  limped  linear 
admittances  Yn(s),  n»l,2,...,N,  having  no  polss 
or  zeroes  in  th*  closed  right  half  plan*.  Then 
th*  network  is  stable,  in  th*  sense  that  it  has  no 
natural  frequency  in  the  doeed  right  half  plane 
except  perhaps  at  th*  origin,  if  at  each  frequency 
w>0  there  exists  a  phase  angle  B(w)  such  that 
0«»(w)<904  and  |£Yn(Jw)  -  «(Jw)|<  f0».  n-1,2 . N. 

Bn  equivalent  statement  of  this  last  condition 
is  that  th*  Wyquitt  plot  of  each  cell's  output  ad¬ 
mittance  for  u>0  never  Intersects  th*  claaed  3rd 
quadrant,  and  that  no  two  call's  output  admittance 
phase  angles  can  ever  differ  by  as  much  am  1B0*. 

It  ell  the  active  cells  are  designed  identically 
and  fabricated  on  the  same  chip,  their  phase  angles 
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should  track  cloaaly  in  practica  and  thus  this 
sacond  condition  is  a  natural  one. 

Mote  that  the  above  stataaent  of  the  theorem 
does  not  rule  out  the  possibility  of  an  unusual 
instability  arising  from  a  repeated  natural  fre¬ 
quency  at  the  origin.  But  a  more  careful  argument, 
omitted  here,  shows  that  the  oily  possible  nonzero 
network  solution  at  *”0  is  the  stable  one  in  which 
capacitors  in  capacitor-only  loops  have  nonzero  d.c. 
voltages  and  all  other  branch  voltages  and  currents 
vanish. 

Proof  of  Theorem  1 


Let  s0  denote  a  natural  frequency  of  the 
network  and  (vk)  denote  the  complex  branch  currents 
at  a  corresponding  solution.  By  Tellegen's  theorem 
[8] ,  or  conservation  of  complex  power ,  we  have 

1 2 r*  _  •  -2  .  r  ...  ■  >2 


I  ivk 


*k''/%  *  I  «cckivk 

resistances  capacitances 


+  I  Vso'vr 

cell 

output  branches 


(2) 


Solutions  of  the  form  s  •  ju  *<  0  can  be  ruled  out 
as  follows.  Note  that  tor  each  <*>0  all  the  cell 
admittance  values  Yn ( j  — )  lie  strictly  above  and  to 
the  right  of  a  straight  line  through  the  origin  of 
the  ccanplex  plane  making  an  angle  of  c(.)  -  90*  with 
the  real  positive  axis.  The  capacitance  admittances 
{juCfc}  and  the  resistor  admittances  {R^-1;  also  lie 
above  and  to  the  right  of  this  line.  Thus  no 
positive  linear  combination  of  these  admittances 
can  vanish  as  required  by  eq.  (2). 

To  rule  out  solutions  in  the  open  right  half 
plane,  it  is  shown  by  a  homotopy  argument  that  the 
existence  of  such  a  solution  isplies  the  existence 
of  a  network  satisfying  the  conditions  of  Thm.  1  and 
having  natural  frequencies  of  the  form  s0  «  *  0 

(already  shown  not  to  exist) .  Add  a  parallel 
conductance  G  to  each  element  of  the  network,  and 
call  the  parallel  element  pair  a  "composite 
element."  Consider  the  locus  of  the  natural  fre¬ 
quencies  as  G  is  increased  from  zero  to  arbitrarily 
high  values.  Eventually  they  must  all  enter  the 
open  left  half  plane  because  all  the  composite  ele¬ 
ments  become  strictly  passive  at  sufficiently  high  G 
values.  Since  the  network  started  out  with  at  least 
one  open  right  half  plane  natural  frequency,  and  the 
natural  frequencies  depend  continuously  on  G,  then 
there  exists  a  G>0  such  that  the  network  has  natural 
frequencies  of  the  form  se  «  ju>  P  0  {sQ-0  is  ruled 
out  by  the  strict  passivity  of  all  the  composite 
elestents  here).  It  is  easily  verified  that  the 
collection  of  composite  network  elements  satisfies 
the  Thm.  1  conditions.  Thus,  open  right  half  plane 
natural  frequencies  are  ruled  out. 


IV.  STABILITY  RESULT  FOR  NETWORKS  WITH  NONLINEAR 
RESISTORS  AND  CAPACITORS 

The  previous  result  for  linear  networks  can 
afford  some  limited  insight  into  the  behavior  of 
nonlinear  networks.  First  the  nonlinear  equations 
are  linearized  about  an  equilibrium  point  and 
Theorem  1  is  applied  to  the  linear  model.  If  the 
linearized  model  is  stable,  then  the  equilibrium 
point  of  the  original  nonlinear  network  is  locally 
stable,  i.e.,  the  network  will  return  to  that 


equilibrium  point  if  the  initial  condition  is 
sufficiently  near  it.  But  the  result  in  this 
section,  in  contrast,  applies  to  the  full  nonlinear 
circuit  model  and  allows  one  to  conclude  that  in 
certain  circumstances  the  network  cannot  oscillate 
even  if  the  initial  state  is  arbitrarily  far  from 
the  equilibrium  point. 

Terminology 

He  sey  that  a  function  y>f(x)  lies  in  the 
sector  [a,b]  if  a»x2  <_  xKfx)  <_  b*x2.  And  we  say 
that  an  Impedance  Z(s)  satisfies  the  Popov  criterion 
it  (1  ♦  rs )S(»i  is  positive  reel  17,9,10)  for  some 
r>0.  Note  that  this  statement  of  the  Popov  criterion 
differs  slightly  from  that  given  in  standard 
references  (9,10) . 

Theorem  2 


Consider  a  network  consisting  of  possibly  non¬ 
linear  resistors  and  capacitors  and  cells  with 
linear  output  izg>edances  z„(s),  n»l,2,...,N.  suppose 

i)  the  resistor  curves  are  continuous  functions 
ik  *  9k(vk*  “bare  gk  lies  in  the  eector 
Gnax>o,  for  all  raiistora, 

ii)  the  capacitora  are  characterized  by 

*k  *  Ck(vk)*k  where  0  <  C^lvjj)  ^  Cm,,  for  all  k  and 
vk ,  and 

iii)  the  impedances  Z^s)  all  satisfy  the  Popov 
criterion  for  seme  comon  value  of  r>0.  Then 
the  network  ie  stable  in  the  sense  that,  for  any 
initial  condition, 

pm 

r  2 

|  l  ik(t)  j  dt  <  -  .  <: 

'“all  resistors  •* 

0  and  capacitora 
Outline  of  Proof 


By  Tellegen's  theorem,  for  any  set  of  initial 
conditions  and  any  time  T>0, 

r  r 

!  I  <vk(t>  +  r  vk(t))ik(t)  dt  ♦ 

• Q  resistors 


l  (vk(t)  +  r  vk(t))ik(t)  dt  ♦ 

J  Q  capacitors 

•T 

I  (V  (t)  ♦  r  V.  (t))i.  (t)  d  t  -  0.  (4) 

.  o  0,11  * 

U  impedances 

For  resistors,, multiplying  the  sector  inequality 
v  g(v)  *  G_.v  by  g  »  0  yields  i2-  ig(v)  <  Grr.i  v. 
and  hence 


ii  r  i  j*  dt 


■T 

ik(t,Ivk(t»  r  Vk(t))dt  -  rl^fv^tn-^IVjjO)) 


S 


where 


♦k<v)  -  j  gk<v')dv'  »  0 


is  ths  resistor  co-con tsnt.  Using  the  inequality 
(6)  in  (5)  yields 

T  T 

g-I- Jlfc It) dt-rSj  <vk (0) ) <  jik<t>  lvk(t)+rvk(t))dt.^ 

Pgr  capacitors,  integrating  the  inequality 
*k  “  ck  (vk>  *£  1  <W\<*k >♦£ 

t  - r  Cc‘‘v5‘,,"K  ■ 

[  i.  <t)  [v.  (t)  ♦  r».  (t)]dt  -  IE.  (q.  (T) -Ek  (qk  (0)1, 
j0  k  k  k  k  k  k  *  (8) 

where 

f<J 

E.  (q)  -  v  (q')dq'  >  0  (9) 

is  the  capacitor  energy.  Using  the  inequality  <9) 
in  (8)  yields 

r~  {ik<t)dt-Ek(qk(0.<  ^(t)  (vk(t)^k(t»)dt. 

m.v  Jn  ' n  nm 


And  for  the  cells,  the  assumption  that  (l«rs)Zn<s) 
is  positive  real  implies  that 

fT 

i  (t)  (v  (t)  ♦  rv  Ct)  1  at  >  -  E  (9),  (11) 

)Q  n  n  n  —  n 

where  E  (0)  is  the  "initial  energy  in  the  cell's 
output  Impedance"  at  t-0,  a  function  of  the  initial 
conditions  only.  Substituting  (7),  (10’  and  (11) 
into  (4)  yields 

<wl  I  ♦  r~  *  I‘k««d'  - 

'0  max  '0 

resistors  capacitors 

r  l  *k<vkC0) )  +  l  Ek(qkIO))  *  l  En (0)  ,  (12) 
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resistors 


capacitors 


where  the  right  hand  side  is  a  function  only  of  the 
initial  conditions.  Thus  (3)  holds.  _ 


V.  CONCLUDING  REMARKS 


The  design  criteria  presented  here  are  simple 
and  practical,  though  at  present  their  validity  it 
restricted  to  linear  models  of  the  cells.  Thsra 
are  several  areas  of  further  work  to  be  pursued, 
one  of  which  is  en  analytic  of  the  differentiator 
cell  that  includea  amplifier  clipping  effects. 

Others  include  the  synthesis  of  a  compensator  for 
the  differentiator  cell,  an  extension  of  the  non¬ 
linear  result  to  include  lapedance  multipliers  other 
than  the  Popov  operator,  and  a  waveform  hounding 
analysis  of  the  network  whirh  would  guarantee 
adequate  convergence  after  an  allotted  settling 
time. 


